Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison

Abstract Assessing goodness of fit to a given distribution plays an important role in computational statistics. The probability integral transformation (PIT) can be used convert the question whether sample originates from reference into problem testing for uniformity. We present new simulation- and optimization-based methods obtain simultaneous confidence bands whole empirical cumulative function (ECDF) PIT values under assumption Simultaneous correspond such intervals at each point that jointly satisfy desired coverage. These also applied cases where is represented only by finite sample, which useful, example, simulation-based calibration. provide intuitive ECDF-based graphical test uniformity, provides useful information on quality discrepancy. further extend simulation optimization determine multiple samples come same underlying distribution. This comparison as complementary diagnostic multi-chain Markov chain Monte Carlo (MCMC) convergence diagnostics, most currently diagnostics single value, but do not usually offer insight nature deviation. numerical experiments assess properties tests using both simulated real-world data give recommendations their practical application statistics workflows.